OMNI: A Framework for Integrating Hardware and Software Optimizations for Sparse CNNs

Convolution neural networks (CNNs) as one of today's main flavor deep learning techniques dominate in various image recognition tasks. As the model size modern CNNs continues to grow, network compression have been proposed prune redundant neurons and synapses. However, prior disconnect software hardware acceleration, which fail balance multiple design parameters, including sparsity, performance, area cost, efficiency. More concretely, unstructured pruning achieve high sparsity at expense extra performance overhead, while structured relying on strict sparse patterns lead low cost. In this article, we propose OMNI, a framework for accelerating accelerators. The innovation OMNI stems from that it uses amenable on-chip memory partition seamlessly engage CNN acceleration. To accelerate compute-intensive convolution kernel, promising optimization approach is partition, divides original weight kernels into several groups so different processing elements can simultaneously access weight. We exploit block, cyclic, or hybrid means patterns. Our balances across our accelerator employs parallelization coordinately with patterns, leading desirable compromise between performance. further develop models help designers quickly identify pattern factors subject an constraint. Last, evaluate application specific integrated circuit (ASIC) field-programmable gate array (FPGA) platform. Experiments demonstrate achieves 3.4×- 6.2× speedup CNNs, over comparably ideal dense accelerator. shows 114.7× energy efficiency improvement compared GPU also evaluated Xilinx ZC706 ZCU102 FPGA platforms, achieving 41.5 GOP/s 125.3 GOP/s, respectively.